Engineers use compression (also called source coding or source encoding) to reduce the bit rate of digital video. Compression decreases the cost of storing and transmitting video information by converting the information into a lower bit rate form. Compression can be lossless, in which case quality of the video when reconstructed does not suffer but decreases in bit rate are limited by the complexity of the video. Or, compression can be lossy, in which case quality of the reconstructed video suffers but decreases in bit rate are more dramatic. Decompression (also called decoding) reconstructs a version of the original information from the compressed form. A “codec” is an encoder/decoder system.
Over the last two decades, various video codec standards have been adopted, including the ITU-T H.261, H.262 (MPEG-2 or ISO/IEC 13818-2), H.263, H.264 (MPEG-4 AVC or ISO/IEC 14496-10), and H.265 (HEVC or ISO/IEC 23008-2) standards, the MPEG-1 (ISO/IEC 11172-2) and MPEG-4 Visual (ISO/IEC 14496-2) standards, and the SMPTE 421M standard. A video codec standard typically defines options for the syntax of an encoded video bitstream, detailing parameters in the bitstream when particular features are used in encoding and decoding. In many cases, a video codec standard also provides details about the decoding operations a decoder should perform to achieve conformant results in decoding. Aside from codec standards, various proprietary codec formats (such as VP8, VP9 and other VPx formats) define other options for the syntax of an encoded video bitstream and corresponding decoding operations.
For encoding, a video source such as a camera or screen capture module typically provides video that is converted to a format such as a YUV 4:4:4 chroma sampling format. A YUV format includes a luma (or Y) component with sample values representing approximate brightness values as well as multiple chroma (or U and V) components with sample values representing color difference values. In a YUV 4:4:4 format, chroma information is represented at the same spatial resolution as luma information.
Many commercially available video encoders and decoders support only a YUV 4:2:0 chroma sampling format. A YUV 4:2:0 format is a format that sub-samples chroma information compared to a YUV 4:4:4 format, so that chroma resolution is half that of luma resolution both horizontally and vertically. As a design principle, the decision to use a YUV 4:2:0 format for encoding/decoding is premised on the understanding that, for typical use cases such as encoding/decoding of natural, camera-captured video content, viewers do not ordinarily notice significant visual differences between video encoded/decoded in a YUV 4:2:0 format and video encoded/decoded in a YUV 4:4:4 format. (Human eyes are less sensitive to changes in color (chroma) compared to changes in brightness or intensity (luma).) The compression advantages for the YUV 4:2:0 format, which has fewer samples per picture, are therefore compelling.
There are some use cases, however, for which video has richer color information and higher color fidelity may be justified. In such use cases, the differences between YUV 4:4:4 and YUV 4:2:0 chroma sampling formats are more easily perceived by viewers. For example, for encoding/decoding of computer screen text content, animated video content with artificial hard-edged boundaries, color text, or certain features of video content more generally (such as scrolling titles and hard-edged graphics, or video with information concentrated in chroma channels), a YUV 4:4:4 format may be preferable to a YUV 4:2:0 format. Some codecs support direct encoding and decoding of video pictures in a YUV 4:4:4 format, but the lack of widespread support for codecs using YUV 4:4:4 formats (especially in terms of hardware implementations) is a hindrance. Other prior approaches providing video with YUV 4:4:4 quality are deficient, for many use case scenarios, in terms of rate-distortion efficiency and computational complexity.